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SUMMARY 

It  is  likely  that  the  future  air  fleet  will  include 
uninhabited  air  vehicles  (UAVs)  that  can  be  controlled 
by  an  operator  in  a remote  location.  Such  a system  will 
require  the  operator  to  experience  the  same  view  as  the 
onboard  camera  to  maintain  control  and  keep  track  of  the 
uninhabited  vehicle.  It  should  be  borne  in  mind  that 
uninhabited  vehicles  are  not  likely  to  be  continuously 
operational  but  deployed  only  when  necessary.  The 
interface  must  therefore  be  intuitive,  as  long  periods  of 
time  could  elapse  between  missions.  The  training  needs 
of  the  operator  should  therefore  be  less  intensive  than 
those  currently  necessary  for  the  manned  aircraft  fleet. 
As  missions  may  employ  a semi-autonomous  mode  of 
operation,  there  is  a requirement  for  transparency 
between  the  system  and  the  operator  inputs.  This  paper 
reports  an  investigation  of  the  utility  of  three  Windows- 
driven  menu  systems  and  four  input  devices. 
Performance  with  a touchscreen,  touchpad,  keyboard 
and  mouse  was  compared  on  a waypoint  re-routing  task. 
It  was  anticipated  that  the  innovative  touchscreen  would 
enhance  performance  when  compared  to  the  more 
conventional  input  methods  of  keyboard  or  mouse.  The 
literature  suggested  that  performance  with  the  touchpad 
would  not  be  optimal.  The  experiment  was  run  in  three 
phases,  each  phase  using  a different  menu  structure. 
Pull-down  menus,  pop-up  menus  and  horizontal  menus 
were  included.  The  results  show  that  in  this  type  of 
scenario,  less  emphasis  should  be  placed  on  the  menu 
system  to  be  used  than  the  input  device,  although  pop-up 
menus  may  be  less  desirable.  The  mouse  and  the 
touchscreen  provide  performance  advantages  in 
comparison  to  the  keyboard  and  the  touchpad. 

1.  INTRODUCTION 

It  is  likely  that  the  future  air  fleet  will  include 
uninhabited  vehicles  that  can  be  controlled  by  an 
operator  in  a remote  location.  Such  a system  will  require 
the  operator  to  experience  the  same  view  as  the  onboard 
camera  to  maintain  control  and  keep  track  of  the  vehicle. 
It  should  be  borne  in  mind  that  uninhabited  vehicles  are 
not  likely  to  be  continuously  operational  but  deployed 
only  when  necessary.  The  interface  must  therefore  be 
intuitive  as  long  periods  of  time  could  elapse  between 


missions.  The  training  needs  of  the  operator  should 
therefore  be  less  intensive  than  those  currently  necessary 
for  the  manned  aircraft  fleet.  As  missions  may  employ  a 
semi-autonomous  mode  of  operation,  there  is  a 
requirement  for  transparency  between  the  system  and  the 
operator  inputs.  This  places  the  emphasis  for  control  on 
the  design  of  an  intuitively  usable  workstation. 
Generally  there  are  three  ways  to  input  information: 
cursor  commands  (mouse  or  touchpad),  point  commands 
(light  pen  or  finger)  and  direct  command  methods  that 
use  keyboard  input  or  voice  input  for  sequence  control 
(Gibbons,  1 992).  This  paper  will  investigate  the  utility 
of  three  Windows-driven  menu  systems  and  four  input 
devices  to  aid  control  of  a remote  vehicle.  Performance 
with  pull-down,  pop-up  and  horizontal  menu  systems, 
and  keyboard,  mouse,  touchpad  and  touchscreen  input 
devices,  will  be  considered. 

1.1  Menus 

Menus  are  an  effective  method  of  interface  design  and 
can  make  interface  operation  easier  for  users.  They  offer 
choices  that  have  only  to  be  recognised  by  the  user, 
rather  than  requiring  the  user  to  learn  and  recall  complex 
syntax  and  command  strings,  thus  using  less  cognitive 
resources  and  facilitating  concentration  on  the  primary 
task  (Schneiderman,  1998).  Menu  systems  can  therefore 
be  particularly  useful  when  intermittent  use  of  a system 
is  likely;  this  lends  itself  to  the  control  of  UAVs.  In 
addition,  they  enable  the  user  to  see  the  range  of 
alternatives  available  (Newman  and  Sproull,  1984). 
Menus  also  lead  to  fewer  errors  than  command-based 
interfaces,  because  they  allow  only  certain  actions  to  be 
performed  (Benbasat  and  Todd,  1993).  Once  a user  has 
indicated  his  menu  choice  via  the  input  device  he  will 
receive  feedback  indicating  what  has  happened,  making 
a menu  interface  appealing  to  users  as  well  as  easy  to 
use.  However,  menu  systems  reduce  user  freedom  as 
choices  are  dictated  by  the  system  (Booth,  1992).  There 
are  many  different  types  of  menu,  but  the  ideal  menu  for 
use  within  a particular  interface  emerges  only  as  a result 
of  careful  interface  design. 

Pull-down  menus  have  top-level  headings  permanently 
displayed  on  the  menu  bar  at  the  top  of  the 
screen/window.  The  user  displays  the  menu  by  “pulling 
down”  or  selecting  an  item  (BS  ISO,  1997).  Further 
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selections  can  then  be  made  from  the  displayed  menu. 
Although  the  top  level  choices  are  always  displayed,  the 
menus  can  obscure  screen  space  when  they  drop  down. 

Pop-up  menus  do  not  have  a permanent  menu  bar 
displayed.  A click  on  the  input  device  “pops  up”  the 
menu  wherever  the  cursor  happens  to  be  at  the  time  (BS 
ISO,  1997).  The  advantages  are  fewer  cursor  (and 
therefore  hand)  movements  as  the  user  does  not  have  to 
keep  moving  the  cursor  to  the  top/bottom  of  the  screen 
to  make  a selection  (Newman  and  Sproull,  1984).  In 
addition,  this  format  saves  screen  space.  However,  as 
the  menu  (and  associated  sub-menus)  may  appear  in 
different  places  there  is  no  chance  for  the  user  to  learn 
the  spatial  location  of  frequently  used  menu  items. 
When  information  is  in  a constant  location,  users  acquire 
expectations  about  where  items  will  appear,  and  this 
gradually  improves  response  times  (Norman,  1991). 

Horizontal  menus  are  the  older  type  of  menu  commonly 
found  on  DOS-based  systems.  One  top-level  menu  item 
and  its  associated  sub-choices  are  displayed  on  a status 
bar  usually  at  the  bottom  of  the  display.  To  change  the 
menu  the  user  selects  the  top  level  to  the  left  of  the 
display  bar  and  all  the  viewable  options  change. 
However,  only  one  mode  can  be  presented  at  any  time, 
requiring  the  operator  to  toggle  between  top-level  menus 
as  appropriate.  One  further  limitation  exists:  there  will 
be  a limit  to  the  number  of  menu  items  that  can  be 
displayed  (Newman  and  Sproull,  1984).  However,  the 
advantage  is  that  these  menus  do  not  obscure  the  screen. 

1.2  Input  Devices 

It  has  been  argued  that  input  devices  such  as 
touchscreens,  mice,  joysticks  and  trackballs  have  an 
advantage  over  the  keyboard  because  they  allow 
manipulation  of  the  screen  content  in  a direct  manner 
(Han,  Jorna,  Miller  and  Tan,  1990).  However,  Gould, 
Green,  Boies,  Meluson  and  Rasamny  (1990)  found  that 
data  entry  could  be  faster  with  the  keyboard  than  other 
devices  if  aided  automatic  string  completion  was  used  on 
a task.  Hence,  it  appears  that  the  nature  and  conditions 
of  the  task  can  determine  the  most  desirable  device.  For 
example,  a mobile  ground-station  for  a UAV  may  have 
different  requirements  from  a fixed-base  command  and 
control  station  headquarters.  The  keyboard  is,  however, 
often  regarded  as  the  most  suitable  device  for  data  entry 
(Gould  etal,  1990). 

The  mouse  is  considered  to  be  quick  and  accurate, 
especially  on  smaller  targets  (Sears  and  Schneiderman, 
1991).  However,  it  does  have  some  disadvantages. 
First,  it  consumes  desk  space,  as  does  its  lead,  which  can 
be  distracting  or  become  entangled  (Brown,  1988). 
Secondly,  it  needs  some  practice  to  acquire  the  skill.  For 
very  long  motions  it  also  requires  the  whole  device  to  be 
picked  up  and  repositioned.  Finally,  it  is  a moving  part, 
which  makes  it  liable  to  “getting  lost”  in  public 
applications,  and  it  is  not  as  rugged  as  a touchscreen  for 
this  type  of  application  (Schneiderman,  1998).  Brown 
(1988)  suggests  that  in  mobile  environments  a mouse 
may  not  be  the  most  suitable  input  device.  Once  again  in 


a mobile  UAV  unit,  these  may  be  important 
considerations.  However,  there  are  some  advantages  in 
the  use  of  a mouse.  During  operation  the  hand  does  not 
obscure  the  screen,  it  is  comfortable  to  use  and  may  be 
the  preferred  input  device  for  click  and  drag 
manoeuvres.  It  is  now  the  industry  standard  and  most 
potential  users  would  already  be  familiar  with  it. 

In  a review,  Schneiderman  (1998)  concludes  that  the 
major  advantage  of  the  touch  tablet  is  that  the  hand  does 
not  obscure  the  screen  as  it  would  in  touchscreen  use  and 
that  it  may  be  more  comfortable  for  the  user.  However, 
it  does  take  up  desk  space.  In  addition  Flan  et  al  (1990) 
found  that  performance  is  very  slow  with  the  tablet  and 
users  do  not  like  it. 

Touchscreens  are  considered  to  be  a natural  pointing 
device  (Sears,  1991).  They  are  reported  to  be  simple  to 
operate  and  easy  to  learn,  with  rapid  performance  and 
potentially  low  error  rates  (Sears  and  Schneiderman, 
1991).  It  has  been  reported  that  the  early  problems  with 
precision  have  now  been  resolved  by  advances  in 
technology  (Schneiderman,  1998).  However, 
touchscreens  are  not  ideal  for  typing  large  amounts  of 
data.  In  his  review,  Schneiderman  (1998)  points  out  that 
touchscreens  are  very  durable  as  they  have  no  moving 
parts,  making  them  ideal  for  military  use.  Gould  et  al 
(1990)  state  that  touchscreens  have  “behavioural 
advantages”.  Less  space  is  required  for  their  use  and 
they  are  also  a very  flexible  interface,  capable  of 
presenting  various  displays  to  suit  the  task,  such  as  icons, 
windows  and  keyboards.  A potential  problem  with 
touchscreens,  however,  is  the  possibility  of  touch  biases. 
Touch  biases  refer  to  differences  between  the  intended 
and  actual  locations  selected.  This  could  also  account 
for  some  of  the  reduced  accuracy  sometimes  seen  with  a 
touchscreen. 

Karat  et  al  (1986)  state  that  improved  performance  seen 
on  touchscreens  may  be  due  to  the  fact  that  mouse  and 
keyboard  operation  involves  more  cognitive  processing, 
whereas  pointing  is  a highly  automated  skill  for  humans. 
Schneiderman  (1998)  points  out  that  mouse  use  requires 
not  only  more  cognitive  processing,  but  also  hand-eye 
co-ordination  when  moving  the  cursor.  Sears  and 
Schneiderman  (1991)  therefore  conclude  that  the 
touchscreen  is  as  fast,  or  faster  than,  the  mouse  except 
for  very  small  targets.  Sears  (1991)  also  suggests  that 
the  touchscreen  allows  “more  natural  selections”  (pp 
265).  This  could  be  relevant  for  UAVs,  as  the  operator 
may  be  under  high  workload,  and  the  lower  the  cognitive 
processing  required  the  better. 

In  general  the  most  appropriate  input  device  seems  to 
depend  on  the  type  and  demands  of  the  task,  the 
operational  environment  and  the  end-users 
(Schneiderman,  1998;  Fernandez  et  al,  1988;  Sears, 
1991).  However,  it  was  anticipated  that  an  innovative 
touchscreen  device  would  enhance  performance  on  a 
waypoint  re-routing  task  when  compared  to  the  more 
conventional  input  methods  of  keyboard  or  mouse. 
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2.  METHOD 

2.1  Participants 

Thirty-six  participants  (26  male,  10  female)  were 
selected  from  an  opportunity  sample  of  Defence 
Evaluation  and  Research  Agency  (DERA)  staff.  They 
were  aged  20  to  34  with  a mean  age  of  25.  They 
reported  normal  or  corrected-to-normal  visual  acuity.  Of 
the  36  participants,  4 were  left-handed  and  32  right- 
handed. 

2.2  Materials 

A demonstration  trial,  a practice  trial  and  two  main  trials 
were  generated  using  a Silicon  Graphics  Onyx2 
computer,  presented  on  a monitor  with  a 47.5  cm  x 
30  cm  visual  screen.  Each  trial  consisted  of  a static  map 
display  with  a pre-determined  set  of  waypoints  that 
indicated  the  planned  route  a UAV  was  to  follow.  The 
demonstration  trial  and  the  practice  trial  consisted  of  five 
tasks:  pre-flight  re-routing;  launch  of  vehicle;  in-flight 
re-routing  due  to  a surface-to-air-missile  (SAM)  zone; 
acknowledgement  and  identification  of  a target;  and, 
finally,  recovery  of  the  vehicle.  For  each  of  the  main 
trials  there  were  eight  tasks,  the  five  tasks  as  before  and 
an  additional  pre-flight  re-route,  in-flight  re-route  and 
target  acknowledgement  and  identification. 

The  task  required  participants  to  carry  out  actions  based 
on  events  as  they  occurred.  The  events  were  signified  by 
audio  messages,  generated  by  a Silicon  Graphics  Sound 
Editor  software  package.  New  waypoints,  targets  and 
launch  and  recovery  points  were  also  indicated  by 
flashing  letters  or  symbols  (as  appropriate)  on  the  screen. 

Participants  were  asked  to  carry  out  actions  using  one  of 
four  input  devices.  The  touch  tablet  used  was  a Cirque 
“Smart  Cat”  touchpad  and  will  therefore  be  referred  to  as 
the  touchpad  within  this  paper.  Three  different  menu 
types  were  used  to  allow  the  participants  to  interact  with 
the  display. 

2.3  Design 

A two-way  mixed  measures  design  was  used.  The 
independent  measures  were  input  device  (four  levels) 
and  menu  type  (three  variables).  Each  participant  was 
presented  with  each  of  the  input  devices:  keyboard, 
mouse,  touchpad  and  touchscreen.  However, 
participants  were  exposed  to  only  one  of  the  menu  types: 
pull-down,  pop-up  or  horizontal. 

The  dependent  measures  were  error  rate  (calculated  as 
the  number  of  times  a task  was  not  correctly  completed), 
response  time  (the  time  taken  for  participants  to  press  a 
response  key)  and  total  task  time  (calculated  as  the 
overall  time  taken  to  complete  a trial).  For  the  purposes 
of  reporting  the  data  the  main  trials  were  broken  down 
into  five  tasks.  Replicated  tasks  were  simply  meaned. 
Subjective  measures  were  also  recorded  in  pre-  and  post- 
experimental  questionnaires. 

Each  participant  was  presented  with  only  one  of  the 
main  trials  (1  or  2)  for  each  input  device.  Main  trial  2 


represented  a reversal  of  events  and  a 180°  rotation  of 
main  trial  1.  To  reduce  familiarity  with  the  route,  for  all 
participants  the  first  input  device  presented  was  shown 
with  main  trial  1,  the  second  with  main  trial  2,  the  third 
with  main  trial  1 and  the  fourth  with  main  trial  2. 

2.4  Procedure 

To  prevent  learning  and/or  order  effects,  the  order  of  use 
of  input  devices  was  balanced  using  a replicated  4x4 
design.  Twelve  participants  completed  each  menu 
system.  Upon  arrival  participants  were  presented  with 
written  instructions  for  completion  of  the  task.  They 
were  asked  to  complete  a demographic  questionnaire 
that  included  questions  regarding  their  familiarity  with 
and  preference  for  the  four  input  devices  and  their 
familiarity  with  the  appropriate  menu  system.  They  were 
then  asked  to  sign  consent  forms.  A practical 
demonstration  was  given  by  the  experimenter,  on  how  to 
complete  the  task,  with  emphasis  on  the  menu  system  to 
be  used.  Following  this,  participants  received  a further 
set  of  instructions  outlining  the  first  input  device  to  be 
used.  Participants  then  completed  one  practice  trial  and 
one  main  trial.  Before  each  of  the  subsequent  input 
devices  was  used,  device-specific  instructions  were 
presented,  but  no  further  demonstrations  were  given. 

During  the  practice  trials,  data  were  not  recorded.  In  the 
main  trials,  response  times  were  recorded  in  seconds  to 
four  decimal  places.  Following  completion  of  the 
experiment,  participants  completed  a post-experimental 
questionnaire  that  asked  them  to  rank  again  their 
preferences  for  the  input  devices.  Participants  were  then 
debriefed. 

3.  RESULTS 

3.1  Objective  Measures 

As  explained  in  the  design  section  of  the  method,  each 
participant  was  presented  with  one  of  the  two  main  trials 
for  each  input  device.  There  is  evidence  of  training 
between  the  two  trial  runs,  whereby  run  1 produced 
significantly  longer  response  times  than  run  2.  However, 
this  does  not  interact  overall  with  the  menu  systems  and 
input  devices  and  will  therefore  not  be  reported  further. 

3.1.1  Error  Rates 

Very  few  errors  were  made  in  the  trials  (0.4%).  An  error 
analysis  was  therefore  not  conducted. 

3.1.2  Response  Times  (RTs) 

All  data  required  a log  transform.  Missing  values  were 
estimated.  Differences  between  input  devices  and  menu 
systems  were  analysed  using  the  Newman-Keuls  range 
test.  Outliers  were  removed  before  analysis  (0.6%).  All 
means  presented  are  back-transformed.  Trials  in  which 
participants  had  responded  incorrectly  were  not  included 
in  the  analyses.  A two-way  mixed  Analysis  of  Variance 
(ANOVA)  was  perfonned  on  the  factors  Menu  and  Input 
Device. 
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3.1.2.1  Pre-flight  re-routing 

A significant  main  effect  of  Input  Device  (F  (3,  99)  = 
45.62,  p<0.001)  was  found.  Post  hoc  analyses  for  the 
main  effect  of  Input  Device  showed  that  the  mouse  and 
touchscreen  produced  significantly  faster  response  times 
than  the  touchpad  (p<0.001)  and  the  keyboard 
(p<0.001).  Mean  RT  scores  for  Input  Device  and  Menu 
are  shown  in  Table  1. 


Input  Device 

Pull-down 

Pop-up 

Horizontal 

Mean 

Keyboard 

21.31 

21.12 

18.07 

20.17 

Mouse 

12.59 

14.84 

13.40 

13.61 

Touchpad 

20.22 

22.61 

20.34 

21.05 

Touchscreen 

13.43 

12.40 

13.32 

13.05 

Mean 

16.89 

17.74 

16.28 

Table  1:  Mean  RT for  Input  Device  and  Menu  for  Pre- 
flight Re-routing  (in  seconds) 


3.1.2.2  Launch  of  the  vehicle 

Significant  main  effects  of  Menu  (F  (2,  125)  = 17.81, 
p<0.001)  and  Input  Device  (F  (3,  125)  = 6.26,  p<0.01) 
were  found.  Post  hoc  analyses  for  the  main  effect  of 
Menu  showed  that  the  horizontal  and  pull-down  menus 
produced  significantly  faster  response  times  than  the 
pop-up  menu  (p<0.01).  Post  hoc  analyses  for  the  main 
effect  of  Input  Device  showed  that  the  touchscreen  and 
mouse  produced  significantly  faster  response  times  than 
the  keyboard  (p<0.001)  and  the  touchpad  (p<0.001). 
Mean  RT  scores  for  Input  Device  and  Menu  are  shown 
in  Table  2. 


Input  Device 

Pull-down 

Pop-up 

Horizontal 

Mean 

Keyboard 

5.10 

5.99 

4.31 

5.14 

Mouse 

3.26 

4.49 

3.88 

3.88 

Touchpad 

4.76 

6.19 

4 96 

5.30 

Touchscreen 

3.68 

3.94 

3.84 

3.82 

Mean 

4.20 

5.15 

4.25 

Table  2:  Mean  RT  for  Input  Device  and  Menu  for  the 
Launch  of  the  Vehicle  (in  seconds) 


3.1.2.3  In-flight  re-routing 

Significant  main  effects  of  Menu  (F  (2,  33)  = 3.65, 
p<0.05)  and  Input  Device  (F  (3,  99)  = 34.63,  p<0.001) 
were  found.  Post  hoc  analyses  for  the  main  effect  of 
Menu  showed  that  the  pull-down  menu  produced 
significantly  faster  response  times  than  the  pop-up 
(p<0.05)  and  horizontal  menus  (p<0.05).  Post  hoc 
analyses  for  the  main  effect  of  Input  Device  showed  that 
the  touchscreen  and  mouse  produced  significantly  faster 


response  times  than  the  keyboard  (p<0.001)  and  the 
touchpad  (p<0.001).  Mean  RT  scores  for  Input  Device 
and  Menu  are  shown  in  Table  3. 


Input  Device 

Pull-down 

Pop-up 

Horizontal 

Mean 

Keyboard 

17.04 

17.41 

17.33 

17.26 

Mouse 

10.86 

13.35 

13.51 

12.57 

Touchpad 

16.91 

18.28 

20.00 

18.40 

Touchscreen 

11.37 

14.12 

14.67 

13.39 

Mean 

14.05 

15.79 

16.38 

Table  3:  Mean  RT for  Input  Device  and  Menu  for  In- 
flight Re-routing  (in  seconds) 


3. 1.2.4  Acknowledgement  and  identification  of  a 
target 

Significant  main  effects  of  Menu  (F  (2,  33)  = 5.98, 
p<0.01)  and  Input  Device  (F  (3,  99)  = 54.60,  p<0.001) 
were  found.  There  was  also  a significant  interaction 
between  Input  Device  and  Menu  (F  (6,  99)  = 2.85, 
p<0.05).  Post  hoc  analyses  for  the  main  effect  of  Menu 
showed  that  the  pull-down  and  horizontal  menus 
produced  significantly  faster  response  times  than  the 
pop-up  menu  (p<0.01).  Post  hoc  analyses  for  the  main 
effect  of  Input  Device  showed  that  the  touchpad 
(p<0.01),  touchscreen  (p<0.001)  and  the  mouse 
(p<0.001)  produced  significantly  faster  response  times 
than  the  keyboard.  The  touchscreen  and  the  mouse  also 
produced  significantly  faster  response  times  than  the 
touchpad  (p<0.001).  In  addition,  the  mouse  produced 
significantly  faster  response  times  than  the  touchscreen 

(p<0.01). 

Post  hoc  analyses  for  the  interaction  between  Input 
Device  and  Menu  showed  that  for  the  pull-down  menu 
the  touchpad  (p<0.05),  the  touchscreen  (p<0.001)  and 
the  mouse  (p<0.001)  produced  significantly  faster 
responses  than  the  keyboard.  The  touchscreen  and  the 
mouse  also  produced  significantly  faster  response  times 
than  the  touchpad  (p<0.001).  For  the  pop-up  menu  the 
touchscreen  and  the  mouse  produced  significantly  faster 
response  times  than  the  keyboard  (p<0.001)  and  the 
touchpad  (p<0.001).  For  the  horizontal  menu  the 
touchscreen  (p<0.05)  and  the  mouse  (p<0.001)  produced 
significantly  faster  response  times  than  the  keyboard. 
The  mouse  also  produced  significantly  faster  response 
times  than  the  touchpad  and  the  touchscreen  (p<0.001). 
For  the  keyboard  the  horizontal  menu  produced 
significantly  faster  response  times  than  the  pop-up  menu 
(p<0.05).  For  the  mouse  and  touchpad  the  pull-down 
and  horizontal  menus  produced  significantly  faster 
response  times  than  the  pop-up  menu  (p<0.01).  Mean 
RT  scores  for  Input  Device  and  Menu  are  shown  in 
Figure  1. 
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Figure  1:  Graph  showing  RTs  for  Input  Device  and  Menu  for  the  Acknowledgement  and  Identification  of  a Target 


(in  seconds) 


3.1.2. 5 Recovery  of  the  vehicle 

Significant  main  effects  of  Menu  (F  (2,  33)  = 3.80, 
p<0.05)  and  Input  Device  (F  (3,  99)  = 7.48,  p<0.001) 
were  found.  Post  hoc  analyses  did  not  show  the  source 
of  the  significant  main  effect  of  Menu.  However,  it  can 
be  seen  in  Table  4 that  there  was  a trend  for  the  pull- 
down and  horizontal  menus  to  produce  faster  response 
times  than  the  pop-up  menu.  Post  hoc  analyses  for  the 
main  effect  of  Input  Device  showed  that  the  touchscreen 
(p<0.05)  and  the  mouse  (p<0.001)  produced 
significantly  faster  response  times  than  the  touchpad. 
The  mouse  also  produced  significantly  faster  response 
times  than  the  keyboard  (p<0.001).  Mean  RT  scores  for 
Input  Device  and  Menu  are  shown  in  Table  4. 


Input  Device 

Pull-down 

Pop-up 

Horizontal 

Mean 

Keyboard 

3.32 

7.64 

2.91 

4.62 

Mouse 

2.38 

4.79 

2.64 

3.27 

Touchpad 

4.11 

7.69 

4.28 

5.36 

Touchscreen 

3.44 

4.38 

3.71 

3.85 

Mean 

3.31 

6.12 

3.39 

Table  4:  Mean  RT for  Input  Device  and  Menu  for 
Recovery  of  the  Vehicle  (in  seconds) 


3. 1.2.6  Total  task  time 

A significant  main  effect  of  Input  Device  (F  (3,  129)  = 
43.84,  p<0.001)  was  found.  Post  hoc  analyses  showed 
that  the  touchscreen  and  the  mouse  produced 
significantly  faster  response  times  than  the  keyboard 
(p<0.001)  and  the  touchpad  (p<0.001).  Figure  2 
illustrates  the  total  task  time. 


Figure  2:  Mean  RT  for  Input  Device  and  Menu  for  Total  Task  Time  (in  seconds) 
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3.2  Subjective  Measures 

3.2.1  Pre-experimental  familiarity  with  input  devices 

Participants  were  asked  to  rate  how  familiar  they  were 
with  each  of  the  four  input  devices  to  be  used,  before 
participating  in  the  trial.  It  can  be  seen  in  Table  5 that 
the  participants  were  more  familiar  with  the  use  of  the 
keyboard  and  the  mouse  than  the  touchpad  and  the 
touchscreen.  Mean  ratings  for  familiarity  with  the  input 
devices  are  shown  in  Table  5. 


Input  Device 

Never 

Used 

Unfamiliar 

Familiar 

Often 

Used 

Keyboard 

0 

0 

2 

34 

Mouse 

0 

0 

2 

34 

Touchpad 

8 

12 

15 

1 

Touchscreen 

4 

12 

19 

1 

Table  5:  Familiarity  with  the  Input  Devices:  Pre- 
experimental  Mean  Ratings 


3.2.2  Pre-experimental  preferences  for  input  devices 

Participants  were  also  asked  to  rank  order  their 
preferences  for  the  four  input  devices  before 
participating  in  the  trial.  It  can  be  seen  in  Table  6 that 
the  participants  ranked  the  keyboard  and  mouse  highly, 
in  line  with  their  familiarity  of  use.  The  mouse  was  the 
preferred  input  device.  The  ranks  shown  represent  35  of 
the  36  participants’  data.  One  participant’s  data  were 
removed  as  the  post-experimental  questionnaire  was 
incorrectly  completed.  Hence  it  could  not  be  used  for 
comparison.  Mean  ranks  for  the  input  devices  are  shown 
in  Table  6. 


Input  Device 

1st 

2nd 

3rd 

4th 

Keyboard 

11 

12 

10 

2 

Mouse 

16 

16 

3 

0 

Touchpad 

1 

2 

3 

29 

Touchscreen 

7 

5 

19 

4 

Table  6:  Pre-experimental  Mean  Ranks  for  the  Input 
Devices 


3.2.3  Pre-experimental  familiarity  for  menu  systems 

Participants  were  also  asked  about  their  familiarity  with 
the  type  of  menu  system  they  were  about  to  use  in  the 
trial.  As  only  one  third  of  participants  were  presented 
with  each  menu  type,  there  were  only  12  participants  in 
each  group.  It  can  be  seen  in  Table  7 that  participants 
were  more  familiar  with  the  pull-down  menu  and  least 
familiar  with  horizontal  menu  systems.  Mean  ratings  for 
familiarity  with  the  input  devices  are  shown  in  Table  7. 


Menu  Type 

Never 

Used 

Unfamiliar 

Familiar 

Often 

Used 

Pull-down 

0 

0 

0 

12 

Pop-up 

0 

1 

1 

10 

Horizontal 

0 

2 

3 

7 

Table  7:  Familiarity  with  the  Input  Devices:  Pre- 
experimental  Mean  Ratings 


3.2.4  Post-experimental  preferences  for  input 
devices 

Participants  were  asked  again  to  rank  order  their 
preferences  for  the  four  input  devices  on  completion  of 
the  trial.  It  can  be  seen  in  Table  8 that  the  participants 
ranked  the  touchscreen  as  the  preferred  device.  It  is 
interesting  to  note  that  the  keyboard  was  ranked  third 
overall.  The  ranks  shown  represent  35  of  the  36 
participants’  data  as  one  participant  gave  equal  ranks  to 
devices,  despite  the  instructions  given  for  completion. 
This  participant’s  data  were  therefore  removed.  Mean 
ranks  for  the  input  devices  are  shown  in  Table  8. 


Input  Device 

1st 

2nd 

3rd 

4th 

Keyboard 

0 

3 

18 

14 

Mouse 

8 

24 

3 

0 

Touchpad 

3 

2 

12 

18 

Touchscreen 

24 

6 

2 

3 

Table  8:  Post-experimental  Mean  Ranks  for  the  Input 
Devices 


4.  DISCUSSION 

For  total  task  time,  pre-flight  re-routing,  launch  of  the 
vehicle,  in-flight  re-routing  and  recovery  of  the  vehicle, 
the  results  show  that  the  mouse  and  the  touchscreen 
produced  performance  advantages  in  comparison  to  the 
keyboard  and  the  touchpad,  on  a waypoint  re-routing 
task.  However,  for  target  acknowledgement  and 
identification,  the  mouse  gave  faster  response  times  than 
the  touchscreen.  The  touchscreen  may  not  have 
provided  performance  advantages  over  the  mouse 
because  it  did  not  prove  to  be  100%  reliable  over  the 
trial.  There  were  some  instances  when  the  touchscreen 
did  not  respond  at  the  initial  press.  However,  this  was 
not  reported  as  a problem  most  of  the  time  and  may 
sometimes  have  been  due  to  user  style.  For  the 
acknowledgement  and  identification  of  a target,  the 
touchpad  provided  performance  benefits  in  comparison 
to  the  keyboard.  For  the  recovery  of  the  vehicle,  the 
mouse  also  gave  performance  advantages  over  the 
keyboard. 

In  designing  the  experiment  it  was  difficult  to  decide 
how  similar  to  keep  the  use  of  each  menu  and  input 
device.  The  aim  was  to  allow  the  advantages  of  each 
menu  and  input  device  to  be  maintained  within  a similar 
design  structure  to  enable  fair  comparisons  to  be  made. 
However,  with  the  keyboard  this  was  particularly 
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difficult  as  the  task  involved  direct  manipulation.  This 
type  of  task  is  inherently  unsuited  to  the  use  of  a 
keyboard.  To  permit  the  input  of  waypoint  data  when 
using  the  keyboard,  it  was  necessary  to  use  the  tab  keys 
for  some  of  the  menu  selections,  but  the  cursor  keys  for 
others.  Participants  were  also  required  to  press  the  space 
bar  rather  than  the  return  key  to  select  menu  options  and 
acknowledge  dialogue  boxes.  In  addition,  keyboard 
inputs  for  the  pull-down  and  pop-up  menus  required 
other  buttons  to  be  pressed  to  display  menu  options.  For 
example,  with  the  pop-up  menu,  the  control  button  had 
to  be  pressed.  This  may  have  contributed  to  the 
performance  advantage  shown  by  the  horizontal  menu  in 
comparison  to  the  pop-up  menu,  when  using  the 
keyboard.  It  is  more  difficult  to  remember  a particular 
key  than  simply  to  make  a selection  with  a mouse.  More 
familiarity  may  be  required  with  the  keyboard  to 
increase  the  speed  of  use.  Effective  use  of  short-cut  keys 
would  help,  but  these  could  be  forgotten  due  to 
intermittent  use. 

It  is  likely  that  the  poor  performance  of  the  touchpad 
could  be  attributed  partly  to  the  speed/distance  ratio  of 
the  scrolling  mechanism  when  movements  of  large 
distances  were  required.  Several  repetitive  movements 
were  required  to  scroll  across  the  screen  from  one  side  to 
the  other.  A further  problem  participants  experienced 
with  the  touchpad  was  deselecting  items  by  accidentally 
tapping  the  pad  when  moving  the  cursor  to  a menu 
option.  Another  possible  explanation  is  the  unfamiliarity 
of  the  participants  with  this  device.  However,  this  could 
not  be  the  only  explanation,  as  participants  were  also 
unfamiliar  with  the  use  of  touchscreens. 

It  is  interesting  to  note  that  the  results  do  not  show  a 
significant  performance  benefit  overall  (total  task  time) 
for  any  of  the  three  menu  systems  used.  However,  for 
the  launch  of  the  vehicle  and  the  acknowledgement  and 
identification  of  a target,  the  results  show  that  the  pull- 
down and  horizontal  menus  produced  faster  response 
times  than  the  pop-up  menu.  It  is  not  surprising  that  the 
pop-up  menu  required  extra  time  to  respond,  as  an 
additional  button  press  is  required  to  enable  the  top-level 
menus  to  be  observed  before  a selection  can  take  place. 
Some  participants  also  found  that  it  was  inconvenient  to 
have  the  display  “pop-up”  near  the  cursor,  blocking  part 
of  the  display.  In  addition,  further  action  would  need  to 
have  been  taken  to  move  the  position  of  the  menu  on  the 
screen  to  a more  suitable  location.  For  in-flight  re- 
routing, however,  the  results  showed  that  the  pull-down 
menu  gave  faster  response  times  than  both  the  pop-up 
and  horizontal  menus. 

The  interaction  between  Input  Device  and  Menu  for 
acknowledgement  and  identification  of  a target  showed 
that,  for  the  pull-down  and  pop-up  menus,  the 
touchscreen  and  mouse  gave  similar  performance 
advantages.  For  the  horizontal  menu  the  mouse 
performed  the  fastest.  One  possible  explanation  is  that  a 
smaller  movement  was  needed  with  the  mouse  to 
identify  the  target  than  for  the  touchscreen,  as  the  finger 
was  usually  removed  from  the  screen  between  selections. 


As  the  buttons  in  this  horizontal  menu  were  adjacent, 
more  effort  would  be  required  in  using  the  touchscreen 
than  the  mouse.  It  is  interesting  to  note  that,  for  the 
keyboard,  the  horizontal  menu  provided  performance 
advantages  over  the  pop-up  menu.  However,  for  the 
mouse  and  touchpad  the  horizontal  and  pull-down  menus 
gave  similar  performance  levels  above  those  for  the  pop- 
up menu.  It  should  be  noted  that  mode  errors  occurred 
more  frequently  with  the  horizontal  menu.  Such  errors 
typically  occurred  between  the  pre-flight  and  the  in- 
flight re-routing  on  the  second  re-route  task.  Once  the 
target  had  been  identified  a change  of  mode  switched  to 
mission  planning.  As  the  dialogue  boxes  to  enter  the 
waypoint  information  were  similar  to  those  in  the  in- 
flight mode  some  participants  did  not  realise  that  they 
were  not  in  navigation  mode.  It  appears  that  performance 
with  the  touchscreen  is  affected  less  by  menu  type  than 
the  remaining  input  devices  (refer  to  Figure  2). 

The  subjective  data  show  that  one  third  of  the 
participants  were  unfamiliar  with  the  use  of  a 
touchscreen  and  a touchpad,  whilst  some  participants 
had  never  used  such  devices  before.  The  majority  of 
participants  were  familiar  with  the  menu  system  to 
which  they  were  exposed,  but  they  were  less  familiar 
with  the  horizontal  menus  than  the  pull-down  and  pop- 
up menus.  This  is  not  unexpected  given  the  use  in 
industry  of  Microsoft  Windows-based  products.  It 
would  have  been  interesting  to  ask  the  participants  for  a 
subjective  preference  for  menu  type.  However,  this  was 
not  possible  as  participants  were  exposed  to  only  one 
menu  system  and  did  not  know  about  the  testing  of 
others  until  they  had  completed  the  experiment. 

A comparison  of  the  pre-experimental  and  post- 
experimental  rank  orders  for  participants’  preferences  for 
the  input  devices  showed  an  interesting  difference. 
Before  the  trial  began,  participants  ranked  the  mouse  as 
the  preferred  input  device  and  the  keyboard  second. 
However,  on  completion  of  the  experiment,  the 
touchscreen  was  ranked  as  the  preferred  input  device  and 
the  mouse  second.  It  is  possible  that  the  difference  in 
pre-  and  post-experimental  rankings  could  be  partly  due 
to  the  specific  scenario  being  used  to  rank  the  devices 
post-experimentally.  Nevertheless,  it  is  useful  to  note 
that  the  majority  of  the  participants  preferred  the 
touchscreen  rather  than  the  mouse  for  this  task,  and  that 
the  keyboard  was  not  placed  high  in  the  ranks. 

Comments  made  during  the  trials  by  the  participants 
were  recorded  for  later  discussion.  One  comment 
referred  to  the  speed  at  which  the  trial  was  run, 
explaining  that  it  was  easy  to  get  distracted  because  the 
trial  was  too  slow.  However,  the  task  had  to  be  run  at  a 
slower  pace  than  envisaged  to  ensure  the  feasibility  of 
participants  being  able  to  complete  the  first  trial  when 
the  task,  input  devices  and  menu  system  would  be 
unfamiliar  to  them.  In  a real-world  scenario,  a similar 
task  would  require  a monitoring  capability,  so  the 
distraction  element  may  have  actually  made  the  task 
more  realistic.  It  was  also  suggested  that  the  mouse 
cursor  should  be  enlarged  and  that  an  accelerated  mouse 
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may  be  more  appropriate  for  the  task.  Other  input 
devices,  such  as  a trackball,  or  the  use  of  direct  voice 
input,  could  have  been  considered.  A further  suggestion 
was  that  some  buttons  could  be  permanently  available  on 
the  screen  for  touchscreen  use;  this  would  reduce  the 
number  of  menu  options  required  on  the  display.  It  was 
recognised  by  some  participants  that  actions  were 
required  only  when  a new  waypoint  marker  had  been 
reached.  Unfortunately  this  was  a consequence  of  the 
way  the  program  was  written;  event  markers  were 
required  to  prompt  activity. 

In  conclusion,  this  research  has  illustrated  the 
significance  that  an  input  device  can  have  for  operator 
performance  on  a waypoint  re-routing  task.  It  appears 
that  in  this  type  of  scenario  less  emphasis  should  be 
placed  on  the  menu  system  to  be  utilised,  although  pop- 
up menus  may  be  less  desirable.  The  focus  should 
therefore  be  placed  on  the  device  for  interaction.  Future 
workstation  design  should  therefore  consider  the  use  of  a 
touchscreen  or  mouse  for  control  of  remotely-operated 
vehicles.  A touchscreen  may  be  preferable  as  its 
performance  seems  more  stable  across  menu  systems  and 
was  reported  as  the  preferred  choice  by  operators  in  this 
experiment.  Future  research  will  investigate  the  utility 
of  the  mouse  and  the  touchscreen  further. 
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